Concerning the problem that images captured by optical system suffer unsteady degradation including noise, blurring and geometric distortion when imaging process is affected by defocusing, motion, atmospheric disturbance and photoelectric noise, a generic framework of image restoration based on natural patch likelihood and sparse prior was proposed. Firstly, on the basis of natural image sparse prior model, several patch likelihood models were compared. The results indicate that the image patch likelihood model can improve the restoration performance. Secondly, the image expected patch log likelihood model was constructed and optimized, which reduced the running time and simplified the learning process. Finally, image restoration based on optimized expected log likelihood and Gaussian Mixture Model (GMM) was accomplished through the approximate Maximum A Posteriori (MAP) algorithm. The experimental results show that the proposed approach can restore degraded images by kinds of blur and additive noise, and its performance outperforms the state-of-the-art image restoration methods based on sparse prior in both Peak Signal-to-Noise Ratio (PSNR) and Structural SIMilarity (SSIM) with a better visual effect.